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Abstract: We discuss the use of multivariate kernel smoothing methods to 
date manuscripts dating from the 11th to the 15th centuries, in the English 
county of Essex. The dataset consists of some 3300 dated and 5000 undated 
manuscripts, and the former are used as a training sample for imputing dates 
for the latter. It is assumed that two manuscripts that are "close", in a sense 
that may be defined by a vector of measures of distance for documents, will 
have close dates. Using this approach, statistical ideas are used to assess "sim- 
ilarity", by smoothing among distance measures, and thus to estimate dates 
for the 5000 undated manuscripts by reference to the dated ones. 



1. Introduction 

The problem of searching for, and comparing, documents on the world wide web 
has motivated the development of techniques for measuring the "relationships" 
among documents. These methods include approaches based on formal measures of 
distance (see e.g. the collections of papers edited by Berry [1, 2] and Djeraba [6]), 
as well as more statistical techniques (see e.g. Cutting et al. [.5]). A brief review of 
the literature, in a statistical context, has been given by Feuerverger et al. [7]. The 
present article builds on the latter paper, by giving relatively detailed accounts of 
the application of kernel-smoothing methods to the dating, or calendaring as it is 
often called, of medieval manuscripts. 

These manuscripts are charters written between the 11th and the 15th centuries. 
They relate to property holdings or transfers in the county of Essex, England. Many 
were taken from entries in the Hospitaller Cartulary of 1442 and are essentially 
land deeds involving the Order of the Hospital of St John of Jerusalem (Gervers 
[11]). The Order, which works internationally today in healthcare, originated in the 
11th century as a monastic brotherhood caring for pilgrims in the Holy Land. The 
dated manuscripts are part of a database assembled at the University of Toronto in 
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connection with the DEEDS (Documents of Essex England Data Set) project, and 
are discussed by Gervers [10, 12, 13]. 

In this paper we interpret the dates of undated documents as missing com- 
ponents of random data vectors of indeterminate length, and impute them using 
nonparamctric, kernel-based regression in which the explanatory variables arc intcr- 
documcnt distances. Sections 2, 3, 4 and 5 respectively discuss measures of distance, 
methodology for smoothing empirical distances, applications of these techniques to 
manuscript data, and theory relating to Section 3. 



2. Shingles, resemblances and distances 

Mathematical formalisation of a manuscript proceeds as follows. Remove all punc- 
tuation from the manuscript. The document that remains is a sequence of n, say, 
words, with repetitions counted as different words. Write this sequence as = 
(wi, . . . ,w„). A consecutive sequence of k words, i.e. S = {wt+i,Wt+2, ■ ■ ■ ,wt+k) 
where Q <.t < n — k, is called a shingle of order k. Let Sk{M.) = {Skij ■ ■ ■ , S'fc^Ar(fc)}, 
where 1 < N{k) < n — fc + 1. denote the set of distinct shingles of order k obtainable 
from the manuscript A4. 

If A4i and Mj are two manuscripts then the mathematical intersection of Sk {M-i) 
and Sk{Mj) is the set of different shingles of order k that are contained in both 
manuscripts. Broder et al. [3] and Broder [4] introduced the notion of the kth order 
resemblance distance, dkii,j), between Mi and Mj. It is the proportion of shingles, 
out of the set of all fcth order shingles in Adi and Aij, that are not contained in 
both A4i and Mj. It can be defined mathematically as 

, , \\SkiM.dnSk{M,)\\ 
dkihj) = 1 — 



||5fc(M0u5fe(X,)ll 

where ||iS|| denotes the number of elements of a finite set S. We shall denote by 
resfc(i, j) = 1 — dk{i,j) the fcth order resemblance. 



3. Smoothing step 

We work with r-vcctors of resemblance distances. In particular, the fcth component 
of the vector describing the distance between Mi and Mj is dk{i,j), for 1 < fc < r. 
Thus, it is assumed that resemblance distances, of orders up to the rth, capture 
the principal ways in which manuscripts differ. There may, however, be significant 
information from other, ordered variables such as document length or the simple fre- 
quencies of certain key words. These could also be incorporated into our smoothing 
algorithm, by making obvious changes to the methodology discussed below. Cate- 
gorical variables such as document "type" , for example whether the document is a 
marriage contract or a land deed (in the context of Latin manuscripts discussed in 
Section 4), are arguably best included by smoothing within the respective type. 

If manuscript Mi is undated, and manuscripts Mj, for j G J^, have respec- 
tive known dates tj, then the kernel weight applied to Mj, based on its nearness 
to Mi, is 

r 

^a{i,j\hi,...,hr) = Y[ K{dk{i,j)/hk}, 

k=l 
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where K denotes a nonnegative, nonincrcasing function defined on the positive 
half-hne, and hi, . . . ,hr are bandwidths. Our estimator, ti of the date ti of Aii is 

k = argmin I ^ {tj - tf a{i,j) > = \ ^tj a{i,j) [ / j °'(*'-^) f ' ^^'^^ 

Theoretical properties of this estimator will be discussed in Section 5. 

There is precedent in the field of manuscript dating for using robust methods. 
In this context one could minimise the loss function J2jeJ ~ ^) where 
\I' is a positive, symmetric, "cup shaped" function with its unique minimum at the 
origin and increasing no faster than linearly in its tails. This leads to the estimator 
t = ti that solves 

4'{tj-t)a{i,j) = 0, 

where ip — Taking 'I'(w) = \u\ we obtain the local median. See Hardle and 
Gasser [14] for discussion of robust nonparamctric methods in a univariate setting. 
To choose bandwidths, define 

ij> =ij'{hi,...,hr) = a.rgmm ^ {tj - tf a{j' J \hi, . . . ,hr) , 

(/ii, ... = argmin ^ {tj/ - £y (/ii, . . . , /i^)}^ , 
{hi.,...,K) 

where IC is the union, over 1 < fc < r, of the set of all j ^ J such that dk{i,j) is 
among the m largest values of that quantity. Here m would give an appropriately 
small fraction of the total number of dated manuscripts. Our empirical choice of 
the bandwidth vector is {hi, . . . , hr). This method is a form of predictive cross- 
validation. See Feuerverger et al. [7] for discussion of refinements. 



4. Application to Latin manuscript data 

4.I. The dataset, and approaches to calendaring 

At the time of writing this paper, the set of dated manuscripts consisted of 3353 
charters written in the 11th, 12th, 13th, 14th and 15th centuries. In addition there 
were some 5000 undated manuscripts. An example of a dated manuscript is given 
in the Appendix. 

The term "dated" above is used in an informal sense, and does not imply that 
a dated manuscript always had an exact date written upon it. Indeed, the major- 
ity of dated manuscripts are calendared by internal evidence, and inaccuracies are 
sometimes present. Gervers [12] details the nature and potential size of these errors; 
discrepancies of the order of several generations are possible. Witness names are 
one source of internal evidence, but identical witness names appear on manuscripts 
that could not possibly have been witnessed by the same person, and sometimes on 
manuscripts dated 100 years or more apart. See, for example, Rees ([15], p. xvii). 
Furthermore, witness names can be truncated, making them hard to identify reli- 
ably; or the names may be omitted altogether. Handwriting evidence can also be 
used for dating, but it too has drawbacks (e.g. Stenton [17], p. xxxii). 

In cases where reliable internal dating is not possible, use can be made of the fact 
that the language, form and content of medieval manuscripts is constantly changing 
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with time (e.g. Stenton [17]). This "dating by formulae" approach, as it is sometimes 
called, is outlined by Gervers [12, 13] in connection with the DEEDS manuscripts 
project. See also Fiallos [8, 9], who describes some algorithms for dating. Both 
Fiallos and Gervers refer to a "shingle" as a "word pattern" or "string" ; the term 
"shingle" was introduced by Broder [4]. The techniques discussed by Gervers are 
substantially more interactive, and more demanding of expert historical knowledge, 
than automated-statistical approaches such as those suggested in the present paper. 
While this may enhance accuracy in some cases, it makes the methodology difficult 
to transfer to other applications. 

4-2. Data analysis 

Figure 1 shows a histogram giving the distributions of dates for the full dataset, of 
size 3353. The dates range from 1089 to 1466, and are seen to be more concentrated 
towards central values. By means of random sampling these documents were divided 
into three disjoint groups. The ffist group, consisting of 3034 documents, served as 
a training set; the second, of 167 documents, was used for validation; and the 
third, of 152 documents, was set aside to serve as a test set. The decision to set 
aside a validation subset, rather than use leave-one-out validation, was made to 
reduce computational labour. Below we report results obtained using resemblance 
distance; see Section 2 for a definition. Throughout we employed the exponential 
kernel, K{x) = . 

We experimented with shingles of sizes one, two and three, as well as with all 
combinations of these sizes. With every such combination we varied the value of 
TO, introduced in Section 3 and defined as the number of "closest" charters to be 
included in the weighted estimation procedure, over the range to = 5, 10, 20 and 
50. Optimisation over h was done by searching over a fine grid. The "shingling" of 
charters into lists of unique sequences of consecutive words, the matching of shingles 
among charters, the computation of distances between them, and the estimation of 
dates, were carried out employing C and standard UNIX commands. 

Using shingle sizes greater than three was ruled out, to limit the amount of 
computation required and also in light of the findings reported below. In this section, 
so as to describe the effects of different shingle sizes, we shall present results in the 
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Fig 1. Histogram of the dates for the complete dataset. 
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setting of univariate smoothing over resemblance distances. That is, we took r = 1 
in the setting of Section 3, although that single component was any one of di{i,j), 
d2{hj) and d3{i,j)] the particular value will be made clear in discussion. Taking 
r = 3 produces results which have properties resembling an "average" of those 
discussed below. 

To convey a broad impression of statistical properties of the data we mention 
that the total number of resemblance values among the validation and training 
manuscripts was 167 x 3034 = 506,678. The mean values were 0.083, 0.014 and 
0.0042 for shingles of size k — 1, 2 and 3, respectively. The pairwise correlations 
between the 506,678 resemblance values, for resemblances based on shingle sizes 
k = 1,2 and 3, were 0.93 between shingle sizes 2 and 3, 0.88 between shingle sizes 1 
and 2, and 0.76 between shingle sizes 1 and 3. Resemblance values can occasionally 
be quite large; they exceeded 0.5 a total of 14, 7 and 4 times among the resemblances 
based on shingle sizes 1, 2 and 3, respectively. Note too that the mean year of the 
training documents was 1245.8. If this value were used as the date estimate for each 
document in the validation set, the mean absolute error would be 36.6 years. 

For shingles of size fc = 1, and using resemblance distance, the optimal value 
of {m,h) was found to be approximately (10,9.0 x 10""^). (The small values for 
bandwidth, here and below, reflect small values of resemblance.) These choices 
resulted in an average absolute difference between true and estimated dates for 
charters, within the validation set, of 13.1 years. When only the closest m ^ 5 
charters were used the average error only increased slightly, to 13.2 years, while 
m = 20 and m = 50 resulted in average absolute errors of 13.2 and 13.3 years, 
respectively, when each was used in conjunction with its corresponding optimal 
bandwidth. 

As can be seen, the results are quite robust against choice of m. Note, however, 
that for a particular value of m, and for a given charter in the validation set, 
it can occasionally happen that there are fewer than m charters (in the training 
set) that have nonzero resemblance to it. When that occurred, only those charters 
having nonzero resemblances were included in the estimation procedure. In this 
sense the effective value of m could occasionally be smaller than its nominal value, 
particularly when m was large. 

The results are also robust against varying h. For example, in the optimal case, 
m = 10, the average error stayed below 13.2 years provided h remained in the 
range (6.9 x 10~^, 1.1 x 10~^). In these instances, as in other results cited below, 
the mean absolute error functions were invariably well behaved as h and m varied, 
and had clearly defined (if somewhat broad) minima. No instances of separated 
multiple minima were discovered during our experimentation. 

Figure 2 shows a grey-scale "image" plot of shingle-size k = 2 resemblances 
between the 167 validation manuscripts (on the vertical axis) and the training 
manuscripts (on the horizontal axis). In producing this plot, the manuscripts on 
each axis were first ordered from earliest date to latest, and resemblance values 
exceeding 0.3 were set equal to 0.3, while values below 0.1 were set equal to zero. 
The training manuscripts were then grouped, with five consecutive manuscripts in 
each of 606 groups, and the resemblance values were averaged for each validation 
manuscript within each training group. These values were then normalized so that 
the maximum value for each validation manuscript was equal to one. Finally, re- 
sulting values at or below 0.8 were set equal to "white," while the value 1.0 was set 
equal to "black;" a linear grey scale was used between these values. The roughly di- 
agonal character of this display mirrors the tendency for documents to have higher 
resemblance values with other documents of comparable dates. The wide scatter 
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Fig 2. Grey-scale plot of shingle-size 2 resemblances between the validation and training 
manuscripts. 



evident in the display also serves to emphasize the inherent difRculty of the problem. 

For shingles of size k ~ 2, and using the resemblance measure, the optimal 
value of {m,h) was (5,6.7 x 10~^), giving a mean absolute error of 11.1 years. 
The mean absolute error stayed below 11.2 years provided h remained in the range 
(5.1 X 10^^,9.5 X lO^'^). When m increased to 10 or to 50 the best mean absolute 
error increased to 11.6 or 11.7 years, respectively. Once again, results are seen to 
be robust against choice of m. 

Analogously, for shingles of size fc = 3 the optimal value of (to, h) was (10, 2.0 x 
10"'^), giving a mean absolute error of 12.1 years. For m = 5 and to = 50 the error 
was 12.4 and 12.2 years, respectively. 

We also experimented with using resemblance measures for pairs of shingle sizes, 
employing bivariate kernels that were products of univariate kernels. We found that 
whenever shingle size fc = 2 was included in a pair the optimisation attempted to 
eliminate the effect of the other shingle size (1 or 3) by assigning to it a relatively 
high (or even infinite) bandwidth. In consequence the results were virtually identical 
to those achieved using shingle size 2 alone: the minimum error achieved using 
shingle sizes 1 and 2 simultaneously was 11.2 years, and was achieved with m — 5, 
while for shingle sizes 2 and 3 together it was 11.1 years and also occurred with 
TO = 5. By way of contrast, for shingle sizes 1 and 3 together the optimal error was 
11.8 years and was attained with (to, hi, h^) = (10, 0.0024, 0.05), where hj denotes 
the bandwidth applied to shingle size j. (Note that when more than one shingle size 
is used, the effective value of to typically is somewhat increased, since the union is 
taken of the to closest documents for each shingle size.) 

The result for using all three shingle sizes (1, 2 and 3) simultaneously was simi- 
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lar to that when using shingle size 2 alone, or using pairs of shingle sizes including 
size 2: minimum mean absolute error was achieved with to = 5, and was 11.2 years. 
Considering this and the previous results, it is seen that the best among the proce- 
dures discussed so far is to use only the resemblance distance based on shingles of 
size 2, taking (m, h) = (5, 6.7 x 10~^). Finally, to obtain independent verification of 
the claimed error rate, this procedure was applied to the 152 documents in the test 
set; the mean absolute error was found to be 12.2 years. The apparent difference 
between the error rates for the validation and test subsets is consistent with the 
bias due to having selected the best of several procedures, with the somewhat small 
sample sizes of these subsets, and with the fact that (due to random sampling) the 
manuscripts in these subsets had slightly different distributions of dates and word- 
counts. Figure 3 shows the true dates for manuscripts in the test set, together with 
their estimated dates. (The zero error line is also superimposed.) Very slight bias 
effects are discernible at the edges due to the fact that for such manuscripts closest 
matches cannot exist at more extreme dates. It is also seen that manuscripts near 
the central date ranges are estimated slightly more accurately, in part owing to the 
availability of many more training manuscripts in the central date ranges (though 
there is also some countervailing influence, since the presence of more documents 
at a certain date slightly biases the selection of closest fitting manuscripts to that 
date). The size of a document being dated was also found to have a modest influ- 
ence on how accurately it could be dated. Very large documents appear to have 
been dated somewhat more accurately than others, but very small documents were 
not, in general, dated inaccurately; indeed, the claimed mean absolute error rate 
appears to be quite generally applicable overall. (Versions of Figure 3 in which dot 
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Fig 3. True and estimated dates for manuscripts in the test set with the zero error line superim- 
posed. 
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sizes varied to reflect manuscript size did not prove to be informative.) 

Finally, we mention that similar detailed numerical studies were also conducted 
for other definitions of distance discussed by Feuerverger et al. [7]. In each of these 
cases, the results obtained were broadly similar to those for resemblance distance, 
with shingles of size 2 alone again being the apparently optimal choice on which 
to base the estimation procedures. For these distances, the optimal mean absolute 
errors obtained all turned out to be slightly larger than that achieved using the 
resemblance distance. 



5. Theoretical properties of kernel imputation 

Let A4o be a particular undated manuscript, written at a time to, and let {Aio, M) 
denote the value of the pair {Mi,Mj) when Mi = Mo and Mj = is a ran- 
domly chosen, dated manuscript. Write Ai for the corresponding value of di(i,j). 
Although the numbers of words in both a manuscript and a dictionary are finite, 
they are potentially so large that values of di{i,j) are virtually distributed in the 
continuum. Therefore it is appropriate to model the joint distribution of the date 
T of a manuscript M, and the distances Ag, by that of a vector (T, Ai, . . . , A^) 
distributed in the contimmm within the region (0, oo) x [0, 1]''. 

In this setting our kernel method for estimating the unknown date io is con- 
sistent if the following two assumptions are satisfied: (a) the mean value of T, 
evaluated when the distances Ai, . . . , A^ are in arbitrarily small neighbourhoods 
of 0, converges to to as those neighbourhoods shrink to 0; and (b) the mean square 
of T, evaluated in the same context, remains bounded. The first of these conditions 
is one of "asymptotic unbiasedness" of the date of a random manuscript M, as 
the distances between M and the fixed manuscript Mo converge to 0. The second 
condition is a common assumption of finite variance. 

Respectively, these two conditions may be stated formally as 

E{TI{A,<Si,...,Ar<Sr)} . 

P{A,<S,,...,Ar<Sr) -'^^ ^^-^^ 

as (5i, . . . , (5r ^ 0, and 

E{T^IiA,<S,,...,Ar<Sr)} 

PiAi<6l,...,Ar<Sr) ^ 

remains bounded as (5i, . . . , (5,- ^ 0. Assume in addition that for each c > 1, 

P(Ai < c<5i,...,cA, < ,5,) 

lim sup . ; T < OO : 

S,,...,S^^O P{Ai<Si,...,Ar<Sr) 

that the kernel K is bounded, continuous, compactly supported and nonincreasing 
on the positive real line; that K{xo) > for some xo > 0; that the dated manuscripts 
{Mj,j £ J^} are independent and identically distributed as M; that the number 
N{J) of elements of is allowed to increase to infinity; and that at the same time 
as N{J) increases, the bandwidths hi, . . . ,hr decrease to 0, but so slowly that 

N{J)P{Ai<hi,...,Ar<hr)-^oo. (5.3) 

Let C denote the set of all conditions stated in this paragraph. 

Theorem 5.1. IfC holds then the estimator ti defined at (2.1) converges in prob- 
ability to the true date of the undated manuscript Mq- 
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Under more refined conditions, properties of bias and variance may be derived. 
In particular it may be shown that variance is generaUy of order {N{J') hi . . . hr}~^ 
and bias of order niax{hi, . . . , h^), and that the estimator is asymptoticaUy Nor- 
maUy distributed. 

To derive the theorem, note that for each e > 0, any kernel K that satisfies 
conditions C may be sandwiched between two kernels Ki and K2, with the prop- 
erties: (a) Ki < K < K2, (b) K2{x) — Ki{x) < e for all x, and (c) Ki and K2 
are each expressible as finite, positive linear combinations of functions of the form 
L{x) ~ 1(0 < X < c), where c > 0. Wc first derive the theorem in the case K ~ L. 

Define A = J2jeJ ^ = ^jej = S = {61,..., 6r) and 

tt{S) = P(Ai < Si,...,Ar < Sr). Then by (5.1) and (5.2) we have in the case 
K = L, 

E{A) = N{J) E{TI{Ai <5i,...,Ar< Sr)} = {<o + o(l)} N{J) 7:{d) , 

var(A) < N{J) E{T^ /(Aj < 5i, . . . , A, < <5,)} = 0{iV( J) ^(,5)} , 

E{B) = N{J) tt{6) and var(B) < N{J) tt{6). It follows from these results and (5.3) 
that A/{N{J)7r{5)} to and B/{N{J) 7r{6)} 1, both convergences being in 
probability. Therefore A/B ^ to in probability, as had to be shown. 

Finally we treat a more general kernel satisfying conditions C. Using the prop- 
erties noted two paragraphs above we may deduce from the results ior K ~ L in 
the previous paragraph that for each e > 0, 

E{B) {to-€ + 0(1)} < E{A) < E{B) {to + e + o(l)} , 

var(A) + var(B) = 0{E{B)} , 
E{B) X N{J) P(Ai <hi,...,Ar<hr). 
Again these results imply the desired property A/B to. 

Appendix A: Typical dated manuscript from database 

Document 00640214, as it appears in the database, is given below. The manu- 
script's date, 1237 AD, is part of its header. All punctuation marks have been 
removed, and numbers (in Roman numerals) are given between exclamation marks. 
Each number is replaced by simply before shingling, so different numbers are 
not distinguished. However, shingling distinguishes capitalised from non-capitalised 
words; for example, "regis" is regarded as different from "Regis" . 

00640214 1237 

Haec est finalis concordia facta in curia domini regis apud Westmonasterium a die 
S Johannis Baptistae in !xv! dies anno regni regis Henrici filii regis Johannis !xxi! 
coram Roberto de Lexinton Willelmo de Eboraco Ada filio Willelmi Willelmo de 
Culewurth justitiariis et aliis domini regis fidelibus tunc ibi praesentibus inter Jo- 
hannem Baioc quaerentem et Robertum Sarum episcopum et capitulum deforciantes 
per Radulfum de Haghe positum loco ipsorum ad lucrandum vel perdendum de advo- 
catione ecclesiae de Waye Bayouse unde assisa ultimae praesentationis summonita 
fuit inter eos in eadem curia scilicet quod praedictus T recognovit advocationem 
praedictae ecclesiae cum pertinentiis esse jus ipsorum episcopi et capituli et eccle- 
siae suae Sarum ut illam quam idem episcopus et capitulum Sarum habent de dono 
Alani de Baiocis patris praedicti Johannis cujus haeres ipse est et idem episcopus 
et capitulum praedictum concesserunt pro se ob successoribus suis eidem Johanni ut 
eidem ecclesiae quotiescunque tota vita ipsius cam vacare contigerit possit idoneam 



330 



A. Feuerverger et al. 



personam praesentare ita quod quicunque pro tempore fuerit persona ejusdem ec- 
clesiae ad praesentationem ipsius Johannis reddet singulis annis praedictis episcopo 
et capitulo sex marcas argenti de praedicta ecclesia apud Sarum nomine pensionis 
scilicet ad festum S Michaelis !xx! solidos ad Natale Domini !xx! solidos ad Pascha 
!xx! solidos ad nativitatem beati Johannis Baptistae !xx! solidos et post decessum 
ipsius Johannis advocatio praedictae ecclesiae cum pertinentiis remanebit praedictis 
episcopo et capitulo Sarum et eorum successoribus quieta de haeredibus ipsius Johan- 
nis in perpetuum Et praeterea idem episcopus et capitulum praedictum concesserunt 
pro se et successoribus suis quod ipsi de caetero invenient unum capellanum divina 
celebrantem singulis diebus anni in capella beati Johannis sita infra parochiam de 
Waye pro anima praedicti Johannis et pro animabus haeredum suorum et anteces- 
sorum suorum et pro cunctis fidelibus in perpetuum et idem episcopus et capitulum 
praedictum et successores sui invenient ornamenta libros et luminaria sufficientia in 
eadem capella in perpetuum 
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